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Claims 

1. An image processing method for determining the movement direction of an object in 
the image pickup range by processing the consecutive images and by following the object 
movement trace present in the images, characterized by the following steps of operation: 

a first step of operation in which all of the consecutive images in a prescribed time are set 
side-by-side in time to form a three-dimensional time-space image; 

a second step of operation in which the time-space image formed in said first step of 
operation is cut by plural planes perpendicular to the original consecutive images, and the images 
appearing on the cross-sections are extracted as cross-sectional images; 

a third step of operation in which the slope of the trace of the object appearing in said 
second step of operation is determined, and the cross-sectional image having the maximum slope 
at the prescribed time is selected; and a fourth step of operation in which the portion at said time 
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of the cross-sectional image selected in said third step of operation is extracted, and the portions 
of the plural cross-sectional images obtained as a result of application of said extracted portion 
on all of the cross-sectional images within the prescribed time are set side-by-side in time 
sequence to form a trace cross-sectional image. 

2. An image processing method for determining the movement direction of an object in 
the image pickup range by processing the consecutive images and by following the object 
movement trace present in the images, characterized by the following steps of operation: 

a fifth step of operation in which the consecutive images from a short time are set 
side-by-side in time sequence to form the time-space image for the short time; 

a sixth step of operation in which the time-space image of said short time is cut by plural 
planes perpendicular to the original consecutive images of said short time, and the images of the 
short time appearing on the cross-sectional plane are extracted as the cross-sectional images; 

a seventh step of operation in which the slope of the object trace appearing on said 
cross-sectional image is determined, and the cross-sectional image of the short time with the 
maximum slope at this time is selected; and an eighth step of operation in which the 
cross-sectional images selected in said seventh step of operation are set side-by-side in time 
sequence to form a trace cross-sectional image. 

3. The image processing method for extracting object movement trace described in Claim 
1 or 2, characterized by the fact that in said second and sixth steps of operation, for each of the 
original consecutive images, after performing edge extraction treatment, theoretical treatment, 
noise removal treatment and other filter treatment and image processing to extract 
characteristics, the plural planes perpendicular to the original consecutive images are considered, 
and the images obtained by projecting said treatment results on said planes are extracted as the 

cross-sectional images. 

4. The image processing method for extracting the object movement trace described in 
any of Claims 1-3, characterized by the fact that said second-fourth steps of operation and said 
sixth-eighth steps of operation include a step of operation in which the cross-sectional images are 
extracted simultaneously when there exist plural objects moving in a single image. 

Detailed explanation of the invention 
[0001] 

Technical field of the invention 

The present invention pertains to an image processing method for extracting an object 
movement trace. Especially, the present invention pertains to an image processing method for 
extracting an object movement trace in view-based image recognition. 
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[0002] 
Prior art 

As a scheme for the moving picture processing with detection of an object moving in a 
three-dimensional space, a scheme has been proposed in which a time-space image prepared by 
setting the consecutive camera images side-by-side in the time axial direction. 

[0003] 

As an example, Taniguchi et al. published a paper with the title of "Proposal of a scheme 
for moving picture processing using time-space image - DTT method - " Denshi Joho Tsushin 
Gakkai Shi, D-II, Vol. J77-D-II, No. 10, pp. 2019-1026, October 1994. In this method, under the 
condition that the movement direction of the object is nearly constant, the useful data pertaining 
to the object with respect to each frame are extracted, and the obtained data are projected in the 
axis (directional axis) parallel to the movement direction of the object so as to form 
one-dimensional data. Then, by setting the one-dimensional data of the various frames 
side-by-side in time sequence, the time-space image is converted to a planar image on the plane 
defined by the directional axis - time axis. For such two-dimensional image, the state of 
movement of the object is represented as one region, and it is possible to detect the movement of 
the object by performing simple two-dimensional image processing. 

[0004] 

Problems to be solved by the invention 

However, in this scheme, it is indispensable to have a schematic knowledge on the 
movement of the object and to set the cutting plane in that direction beforehand. Consequently, 
although there is no trouble when handling an object moving linearly on the image, it is 
nevertheless impossible to handle the general object that moves in two dimensions. 

[0005] 

Consequently, the main objective of the present invention is to provide an image 
processing method for extracting the object movement trace, characterized by the fact that there 
is no need to judge the identity of the object with images taken at different times, the precision in 
following the object trace can be improved, and the resistance to noise improves. 

[0006] 

Means to solve the problems 

The invention in Claim 1 provides an image processing method for extracting an object 
movement trace, characterized by the fact that in the image processing method for determining 
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the movement direction of an object in the image pickup range by processing consecutive images 
and by following the object movement trace present in the images, the operation is performed as 
follows: all of the consecutive images over a prescribed time are set side -by-side in time to form 
a three-dimensional time-space image; the time-space image formed in the aforementioned 
operation is cut by plural planes perpendicular to the original consecutive images, and the images 
appearing on the cross-sections are extracted as cross-sectional images; the slope of the trace of 
the object appearing in said operation is determined, and the cross-sectional image with the 
maximum slope at the prescribed time is selected; and the portion at said time of the 
cross-sectional image selected in said operation is extracted, and the portions of the plural 
cross-sectional images obtained as a result of application of said extracted portion on all of the 
cross-sectional images within the prescribed time are set side-by-side in time sequence to form a 
trace cross-sectional image. 

[0007] 

The invention in Claim 2 provides an image processing method similar to that described 
in Claim 1, and has the following operation: the consecutive images over a short time are set 
side-by-side in time sequence to form the time-space image of that short time; the time-space 
image of said short time is cut by plural planes perpendicular to the original consecutive images 
of said short time, and the images of the short time appearing on the cross-sectional plane are 
extracted as the cross-sectional images; the slope of the object trace appearing on said 
cross-sectional images is determined, and the cross-sectional image of the short time with the 
maximum slope at this time is selected; and the cross-sectional images selected in said operation 
are set side-by-side in time sequence to form a trace cross-sectional image. 

[0008] 

The invention described in Claim 3 pertains to the image processing method for 
extracting object movement trace described in Claim 1 or 2, characterized by the fact that in said 
operation, for each of the original consecutive images, after performing edge extraction 
treatment, theoretical treatment, noise removal treatment and other filter treatment and image 
processing to extract characteristics, the plural planes perpendicular to the original consecutive 
images are considered, and the images obtained by projecting said treatment results on said 
planes are extracted as the cross-sectional images. 

[0009] 

The invention described in Claim 4 is characterized by the fact that the cross-sectional 
images are extracted simultaneously when there exist plural objects moving in a single image. 



[0010] 

Embodiment of the invention 

Figure 1 is a flow chart illustrating the flow of the overall processing in an application 
example of the present invention. Figure 2 is a diagram illustrating the state of the consecutive 
images and movement object. Figure 3 is a diagram illustrating the time-space image. Figure 4 is 
a diagram illustrating the cross-section of the time-space image taken by a plane parallel to the 
time axis. Figure 5 is a diagram illustrating the method for preparing the trace cross-sectional 
image from the plural cross-sectional images. 

[0011] 

As shown in Figure 1, for example, a camcorder is used to take the consecutive images 
that are input to an image processor. Then, in step (referred to as SP) SP1 shown in Figure 1, as 
the images at an instant (1 1, 12, 13) shown in Figure 2 are represented as I(x, y) with the 
orthogonal coordinates of X-axis and Y-axis, all of the images obtained are set side-by-side in 
time sequence. That is, suppose there is a time axis (T-axis) perpendicular to both X-axis and 
Y-axis, by setting the images along this axis, it is possible to construct the three-dimensional 
image shown in Figure 3, that is, time-space image I(x, y; t). In step SP2, said time-space image 
I(x, y; t) is cut by a plane parallel to the time axis, and the image appearing on the cut plane is 
taken as cross-sectional image C(d, t; 9). Here, angle 9 is the angle formed between the plane 
parallel to the time axis and the X-axis of the original image, with 9 = arctan (y/x), and 0 < 9 < n. 
Now, as angle 9 is changed, plural cross-sectional images C(d, t; 9) can be obtained. 

[0012] 

On said cross-sectional image C(d, t; 9), a portion of the image of moving object (21) 
contained in the original consecutive images must appear as the trace. Consequently, in step SP3, 
slope dd/dt on the object trace on said cross-sectional image C(d, t; 9) at a certain time is 
determined, and it is a function of angle 9, and it reaches maximum at a certain angle 9 d (t). In 
step SP4, this is applied to all of the times, and, as shown in Figure 5, only the portion containing 
angle 9 d is extracted from each cross-sectional image C(d, t; 9), and a new image is formed. This 
is called trace cross-sectional image L(s, 9; t), and it is found that this image becomes a place 
containing the velocity vector of the moving object. That is, this trace cross-sectional image L(s, 
9; t) is an image obtained by cutting said time-space image I(x, y; t) with a helix plane along the 
movement direction of the object. This plane completely contains the information pertaining to 
the movement of the object, that is, movement velocity and movement direction. 
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[0013] 

Said application example explains the case when all of the consecutive images within a 
prescribed time are obtained beforehand. However, this is not a necessity. It is also possible to 
execute a series of operations for the consecutive images during an extremely short time, and 
setting the obtained results finally in time sequence to obtain the image corresponding to the 
trace cross-sectional images. 

[0014] 

Said application example explains the method in which the time-space image is simply 
cut by planes so as to obtain plural cut cross-sectional images. However, in order to better 
represent the movement trace of the object on the cross-sectional images, one may also adopt a 
scheme in which edge detection, noise removal, threshold treatment, other filter treatment, and 
similar characteristic quantity extracting treatment are performed on the original consecutive 
images beforehand to form a time-space image, and the results of the aforementioned treatment 
are projected on the plane instead of cutting with a plane, and images identical to the 
cross-sectional images can be obtained, and they can be used in forming the cross-sectional 
image. 

[0015] 

In said explanation, there is only one moving object. When there are plural moving 
objects in the cross-sectional images, plural groups of the cross-sectional images are prepared 
corresponding to the respective objects, to get the trace cross-sectional image for each object. In 
this way, the aforementioned procedure can be adopted as is when there are plural moving 
objects. 

[0016] 

Effects of the invention 

As explained above, according to the present invention, it is alright that the movement 
direction of the moving body as the object is not known beforehand, and one may simply adopt 
an appropriate scheme in selecting the cutting plane. That is, cutting of the initial time-space 
image is performed in all directions. From the cutting results, the optimum cutting direction is 
known, and only its cutting plane is left. As a result, the cutting plane is made of a helix plane 
along the movement trace of the object, and this plane completely contains the movement vector 
of the object, and it contains all of the information about the movement trace. Consequently, the 
scheme for cutting the time-space image has the shared general properties, and the resistance to 
noise is improved. 
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Brief description of the figures 

Figure 1 is a flow chart illustrating the flow of the overall treatment in an application 
example of the present invention. 

Figure 2 is a diagram illustrating the state of the consecutive images and the moving 

body. 

Figure 3 is a diagram illustrating the time-space image. 

Figure 4 is a diagram illustrating the cross-section of the time-space image formed by a 
place parallel to the time axis. 

Figure 5 is a diagram illustrating the method for forming the trace cross-sectional image 
from plural cross-sectional images. 

Explanation of symbols 

11, 12, 13 Consecutive images 

2 1 Moving body 

I Time-space image 

C Cross-sectional image 

L Trace cross-sectional image 
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Key: a Consecutive images 

b Movement velocity, movement direction 

SP 1 Formation of time-space image 

SP2 Formation of cross-sectional image 

SP3 Detection of maximum slope 

SP4 Formation of trace cross-sectional image 
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Figure 2 



Key: 21 Moving body 
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Figure 3 

Key: I Time-space image 
2 1 Moving body 




Figure 4 

Key: C Cross-sectional image 




Figure 5 

Movement trace 
Cross-sectional image 
Trace cross-sectional image 



